Content stream processing

ABSTRACT

In a method for content stream processing, a coded content stream of a content stream is received from a server. Second trigger timing information of the coded content stream is obtained. The second trigger timing information is determined based on first trigger timing information of the content stream and stream time change information. The stream time change information indicates a timing change between the coded content stream and the content stream. The first trigger timing information indicates a target event in the content stream. The coded content stream is decoded. The decoded content stream is played back. When the decoded content stream is played to the target event indicated by the second trigger timing information, specified processing corresponding to the target event is performed.

RELATED APPLICATIONS

The present application is a continuation of International ApplicationNo. PCT/CN2022/131881 filed on Nov. 15, 2022, which claims priority toChinese Patent Application No. 202210117818.6 filed on Feb. 8, 2022. Theentire disclosures of the prior applications are hereby incorporated byreference.

FIELD OF THE TECHNOLOGY

This disclosure relates to an information processing technology in thefield of computer applications, including to multimedia streamprocessing methods and apparatuses, devices, a computer-readable storagemedium, and a computer program product.

BACKGROUND OF THE DISCLOSURE

During processing of multimedia information such as audios and videos,it may be necessary to mark an event moment in the multimediainformation on a server device and transmit the marked event moment to aplayer device, so that the player device triggers specified processingwhen the multimedia information is played to the event moment. Forexample, when a “virtual penta kill” moment is marked on the serverdevice, and the marked “virtual penta kill” moment is transmitted to theplayer device, so that the player device triggers presentation of avirtual red packet when a game is played to the “virtual penta kill”moment.

In order to enable the player device to obtain the event moment, theserver device may directly send the marked event moment to the playerdevice. However, since the multimedia information received by the playerdevice is coded by the server device, the coding of the multimediainformation could change a time stamp corresponding to frame informationof the multimedia information. Therefore, time, corresponding to theevent moment obtained by the player device, in the coded multimediainformation may no longer be a moment at which an event occurs, whichaffects the accuracy of an event occurrence moment in the codedmultimedia information, thereby affecting the accuracy of triggeringspecified processing.

SUMMARY

Embodiments of this disclosure provide content stream processing methodsand apparatuses, devices, a non-transitory computer-readable storagemedium, and a computer program product, which can improve the accuracyof triggering specified processing.

Technical solutions in the embodiments of this disclosure can includethe following implementations.

The embodiments of this disclosure provide a method for content streamprocessing. The method is performed by a player device, for example. Inthe method for content stream processing, a coded content stream of acontent stream is received from a server. Second trigger timinginformation of the coded content stream is obtained. The second triggertiming information is determined based on first trigger timinginformation of the content stream and stream time change information.The stream time change information indicates a timing change between thecoded content stream and the content stream. The first trigger timinginformation indicates a target event in the content stream. The codedcontent stream is decoded. The decoded content stream is played back.When the decoded content stream is played to the target event indicatedby the second trigger timing information, specified processingcorresponding to the target event is performed.

The embodiments of this disclosure further provide a method for contentstream processing. The method is performed by a server device, forexample. Event marking is performed on a content stream to obtain firsttrigger timing information of a target event in the content stream. Thecontent stream is coded to obtain a coded content stream. Stream timechange information that indicates a timing change between the codedcontent stream and the content stream is obtained. The coded contentstream is transmitted based on the first trigger timing information andthe stream time change information, second trigger timing information ofthe target event in the coded content stream being determined based onthe first trigger timing information and the stream time changeinformation.

The embodiments of this disclosure provide an information processingapparatus, such as a first content stream processing apparatus. Theinformation processing apparatus includes processing circuitry that isconfigured to receive a coded content stream of a content stream from aserver. The processing circuitry is configured to obtain second triggertiming information of the coded content stream. The second triggertiming information is determined based on first trigger timinginformation of the content stream and stream time change information Thestream time change information indicating a timing change between thecoded content stream and the content stream. The first trigger timinginformation indicates a target event in the content stream. Theprocessing circuitry is configured to decode the coded content stream.The processing circuitry is configured to play back the decoded contentstream. The processing circuitry is configured to perform, when thedecoded content stream is played to the target event indicated by thesecond trigger timing information, specified processing corresponding tothe target event.

The embodiments of this disclosure provide an information processingapparatus, such as a second content stream processing apparatus. Theinformation processing apparatus including processing circuitry that isconfigured to perform event marking on a content stream to obtain firsttrigger timing information of a target event in the content stream. Theprocessing circuitry is configured to code the content stream to obtaina coded content stream. The processing circuitry is configured to obtainstream time change information that indicates a timing change betweenthe coded content stream and the content stream. The processingcircuitry is configured to transmit the coded content stream based onthe first trigger timing information and the stream time changeinformation. Second trigger timing information of the target event inthe coded content stream is determined based on the first trigger timinginformation and the stream time change information.

The embodiments of this disclosure provide a player device, including afirst memory and a first processor. The first memory is configured tostore computer-executable instructions. The first processor isconfigured to implement, when executing the computer-executableinstructions stored in the first memory, the method provided in theembodiments of this disclosure and applied to the player device.

The embodiments of this disclosure provide a server device, including asecond memory and a second processor. The second memory is configured tostore computer-executable instructions. The second processor isconfigured to implement, when executing the computer-executableinstructions stored in the second memory, the method provided in theembodiments of this disclosure and applied to the server device.

The embodiments of this disclosure provide a non-transitorycomputer-readable storage medium, storing instructions which whenexecuted by a processor cause the processor to perform any of themethods for content stream processing.

The embodiments of this disclosure provide a computer program product,including a computer program or computer-executable instructions, thecomputer program or computer-executable instructions, when executed by aprocessor, implementing any of the methods for content streamprocessing.

The embodiments of this disclosure can at least have the followingbeneficial effects. Among the at least one event triggering momentobtained by the player device, each event triggering moment is obtainedby adjusting the at least one event marking moment on the basis of thestream time change information, and the stream time change informationrefers to the time change before and after the coding of the contentstream. Therefore, each event triggering moment obtained by the playerdevice is a moment at which an event in a to-be-decoded live streamoccurs. In this way, the event triggering moment is accurate in theto-be-played content stream obtained by decoding the to-be-decodedcontent stream. Thus, in the process of playing the to-be-played contentstream, a played content matches the event triggering moment.Accordingly, if any event triggering moment to which the to-be-playedcontent stream is played is just a moment of playing the event content,at this time, the specified processing is triggered, so that theaccuracy of triggering specified processing can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an architecture of a first multimediastream processing system provided according to an embodiment of thisdisclosure.

FIG. 2 is a schematic diagram of an architecture of a second multimediastream processing system provided according to an embodiment of thisdisclosure.

FIG. 3 is a schematic diagram of a structure of a terminal in FIG. 2according to an embodiment of this disclosure.

FIG. 4 is a schematic diagram of a structure of a server in FIG. 2according to an embodiment of this disclosure.

FIG. 5 a is a flowchart of a multimedia stream processing methodprovided according to an embodiment of this disclosure.

FIG. 5 b is a flowchart of an exemplary multimedia stream processingmethod according to an embodiment of this disclosure.

FIG. 6 a is a flowchart of a multimedia stream processing methodprovided according to an embodiment of this disclosure.

FIG. 6 b is a flowchart of adjusting at least one event marking momentaccording to an embodiment of this disclosure.

FIG. 7 a is a flowchart of a multimedia stream processing methodprovided according to an embodiment of this disclosure.

FIG. 7 b is a schematic diagram of obtaining stream time changeinformation according to an embodiment of this disclosure.

FIG. 8 is a schematic diagram of an exemplary event picture according toan embodiment of this disclosure.

FIG. 9 is a schematic diagram illustrating that an exemplary time stampchanges according to an embodiment of this disclosure.

FIG. 10 is a schematic diagram of an architecture of a live streamsystem according to an embodiment of this disclosure.

FIG. 11 is a schematic diagram of a coding result of an exemplary livestream according to an embodiment of this disclosure.

FIG. 12 is a schematic diagram of an exemplary application scenario ofobtaining different live streams according to an embodiment of thisdisclosure.

FIG. 13 is a schematic diagram of an exemplary live stream interactionaccording to an embodiment of this disclosure.

DESCRIPTION OF EMBODIMENTS

To make the objectives, technical solutions, and advantages of thisdisclosure clearer, the following describes this disclosure in furtherdetail with reference to the accompanying drawings. The describedembodiments are not to be considered as a limitation to this disclosure.Other embodiments are within the scope of this disclosure.

In the following description, the term “some embodiments” describessubsets of all possible embodiments, but it may be understood that “someembodiments” may be the same subset or different subsets of all thepossible embodiments, and can be combined with each other withoutconflict.

In the following description, the term “first\second\third” is merelyused to distinguish similar objects and does not represent a specificorder of the objects. It may be understood that “first\second\third” canbe interchanged in a specific order or precedence where permitted, toenable the embodiments of this disclosure described herein to beimplemented in a sequence other than that illustrated or described here.

The use of “at least one of” or “one of” in the disclosure is intendedto include any one or a combination of the recited elements. Forexample, references to at least one of A, B, or C; at least one of A, B,and C; at least one of A, B, and/or C; and at least one of A to C areintended to include only A, only B, only C or any combination thereof.References to one of A or B and one of A and B are intended to include Aor B or (A and B). The use of “one of” does not preclude any combinationof the recited elements when applicable, such as when the elements arenot mutually exclusive.

Unless otherwise defined, all technical and scientific terms used in theembodiments of this disclosure have the same meaning as commonlyunderstood by those skilled in the art of this disclosure. Terms used inthe embodiments of this disclosure are merely intended to describeobjectives of the embodiments of this disclosure, but are not intendedto limit this disclosure.

Before the embodiments of this disclosure are further described indetail, an exemplary description is made on nouns and terms in theembodiments of this disclosure, and the nouns and terms in theembodiments of this disclosure are applicable to the followingexplanations.

1) A client may include an application program run on a device toprovide various services, for example, an instant messaging client, avideo client, and a livestreaming client. A player device may refer to adevice running a client configured to play a multimedia stream. A serverdevice may refer to a backend server that interacts with a player deviceand provides computing services to the player device.

2) Cloud computing may include a computing mode that distributescomputing tasks to a resource pool composed of a large number ofcomputers to enable various application systems to obtain computingpower, storage spaces, and information services as needed. A networkthat provides resources for the resource pool may be referred to as“cloud”. Users may think that resources in the “cloud” can be infinitelyexpanded and can be obtained at any time, used on demand, expanded atany time, and paid for according to use. In this embodiment of thisdisclosure, the server device may be a cloud device.

3) Artificial Intelligence (AI), for example, involves a theory, amethod, a technology, and an application system that use a machinecontrolled by a digital computer to simulate, extend, and expand humanintelligence, sense an environment, obtain knowledge, and use theknowledge to obtain an optimal result. In the embodiments of thisdisclosure, event marking can be performed in combination withartificial intelligence, such as determining a triggering moment ofspecified processing on the basis of artificial intelligence.

During processing of content information, such as multimedia information(for example, a multimedia stream) including audio and/or video, it maybe necessary to mark an event moment in the multimedia information on aserver device and transmit the marked event moment to a player device,so that the player device triggers specified processing when themultimedia information is played to the event moment.

In order to enable the player device to obtain the event moment, theserver device may directly carry the event moment in SupplementalEnhancement Information (SEI) of frame information of the multimediainformation, and code the SEI, so that the player device obtains theevent moment by means of decoding the SEI. However, in the process ofobtaining the event moment by the player device, the server device needsto support coding of the SEI, and the player device needs to supportdecoding of the SEI. This only applies to devices that support codingand decoding of the SEI, which affects the universality.

In addition, when the server device carries the event moment in the SEIof the multimedia information, a size of the multimedia information isincreased. This will increase a bit rate of the multimedia informationand affect the resource consumption of a transmission network.Furthermore, after decoding the SEI to obtain the event moment, theplayer device may need to transmit the event moment to an applicationlayer through a callback function, which reduces resource consumptionand affects the fluency of playing of the multimedia information.Moreover, there may be a situation where the player device cannot invokethe callback function during the decoding of the SEI (such as without aninterface for the callback function). In this way, the application layerwill not be able to obtain the event moment and cannot achievetriggering of the specified processing based on the event moment. Thereare still decoding errors, causing abnormal playing of the multimediainformation. Thus, the stability of triggering specified processing isaffected.

In addition, if the server device directly transmits the marked eventmoment to the player device, since coding the multimedia information bythe server device may change a presentation time stamp corresponding tothe frame information of the multimedia information, a moment,corresponding to the received event moment, in the decoded multimediainformation may no longer be the moment at which the event occurs. As aresult, the marked event moment can fail, which affects the accuracy oftriggering specified processing.

Embodiments of this disclosure provide multimedia stream processingmethods and apparatuses, devices, a computer-readable storage medium,and a computer program product, which can improve the stability,accuracy, and universality of triggering specified processing. Thefollowing illustrates exemplary applications of a device provided in theembodiments of this disclosure. The device provided in the embodimentsof this disclosure may be implemented as various types of terminals suchas a smartphone, a smartwatch, a laptop, a tablet, a desktop computer, asmart appliance, a set-top box, a smart vehicle-mounted device, aportable music player, a personal digital assistant, a specializedmessaging device, an intelligent voice interaction device, a portablegaming device, and a smart speaker, and may also be implemented as aserver. An exemplary application will be explained when the playerdevice is implemented as a terminal and the server device is implementedas a server.

Referring to FIG. 1 , FIG. 1 is a first schematic diagram of anarchitecture of a multimedia stream processing system provided accordingto an embodiment of this disclosure. As shown in FIG. 1 , in order tosupport a multimedia stream application, in the multimedia streamprocessing system 100, a terminal 400 (referred to as a player device,exemplarily showing a terminal 400-1 and a terminal 400-2) is connectedto a server 200 (referred to as a server device) through a network 300.The network 300 may be a wide area network, a local area network, or acombination of the wide area network and the local area network. Inaddition, the multimedia stream processing system 100 also includes adatabase 500 used for providing a data support for the server 200.Moreover, FIG. 1 shows a situation where the database 500 is independentof the server 200. In addition, the database 500 may also be integratedin the server 200. This embodiment of this disclosure does not limitthis.

The terminal 400 is configured to receive, via the network 300, ato-be-decoded multimedia stream, stream time change information, and atleast one event marking moment transmitted by the server 200, and adjustthe at least one event marking moment on the basis of the stream timechange information to obtain at least one event triggering moment, thestream time change information referring to a time change before andafter coding of a multimedia stream, and each event marking momentreferring to an event moment in the multimedia stream before coding;decode the to-be-decoded multimedia stream to obtain a to-be-playedmultimedia stream, and play the to-be-played multimedia stream; andperform, in a case that the to-be-played multimedia stream is played toa target event triggering moment, specified processing (for example,displaying a virtual resource packet on an interface of the terminal400-1 or the terminal 400-2) corresponding to the target eventtriggering moment, the target event triggering moment being any one ofthe at least one event triggering moment.

The server 200 is configured to perform event marking on a to-be-codedmultimedia stream to obtain the at least one event marking moment; codethe to-be-coded multimedia stream to obtain the to-be-decoded multimediastream; obtain the stream time change information between theto-be-coded multimedia stream and the to-be-decoded multimedia stream;and transmit the at least one event marking moment, the stream timechange information, and the to-be-decoded multimedia stream to theterminal 400 via the network 300, so that the terminal 400 adjusts theat least one event marking moment on the basis of the stream time changeinformation to obtain at least one event triggering moment, plays theto-be-played multimedia stream by decoding the to-be-decoded multimediastream, and performs, when the to-be-played multimedia stream is playedto the target event triggering moment, the specified processingcorresponding to the target event triggering moment.

Based on FIG. 1 , referring to FIG. 2 , FIG. 2 is a second schematicdiagram of an architecture of a multimedia stream processing systemprovided according to an embodiment of this disclosure. As shown in FIG.2 , a difference from FIG. 1 is that an interaction process between theterminal 400 and the server 200 is shown.

The terminal 400 is configured to receive, via the network 300, ato-be-decoded multimedia stream transmitted by the server 200 and atleast one event triggering moment corresponding to the to-be-decodedmultimedia stream, the at least one event triggering moment beingobtained by adjusting at least one event marking moment by the server200 on the basis of stream time change information, the stream timechange information referring to a time change before and after coding ofa multimedia stream, and each event marking moment referring to an eventmoment in the multimedia stream before coding; play a to-be-playedmultimedia stream by decoding the to-be-decoded multimedia stream; andperform, in a case that the to-be-played multimedia stream is played toa target event triggering moment, specified processing corresponding tothe target event triggering moment, the target event triggering momentbeing any one of the at least one event triggering moment.

The server 200 is configured to perform event marking on a to-be-codedmultimedia stream to obtain the at least one event marking moment; codethe to-be-coded multimedia stream to obtain the to-be-decoded multimediastream; obtain the stream time change information between theto-be-coded multimedia stream and the to-be-decoded multimedia stream;adjust the at least one event marking moment on the basis of the streamtime change information to obtain the at least one event triggeringmoment; and transmit the at least one event triggering moment and theto-be-decoded multimedia stream to the terminal 400 via the network 300,so that the terminal 400 plays the to-be-played multimedia stream bydecoding the to-be-decoded multimedia stream, and performs, when theto-be-played multimedia stream is played to the target event triggeringmoment, the specified processing corresponding to the target eventtriggering moment, the target event triggering moment being any one ofthe at least one event triggering moment.

In some embodiments, the server 200 may be an independent physicalserver, or may be a server cluster or a distributed system formed by aplurality of physical servers, or may be a cloud server that providesbasic cloud computing services such as cloud services, cloud databases,cloud computing, cloud functions, cloud storage, network services, cloudcommunication, middleware services, domain name services, securityservices, Content Delivery Networks (CDNs), big data, and artificialintelligence platforms. The terminal 400 may be, but is not limited to,a smartphone, a smartwatch, a laptop, a tablet, a desktop computer, asmart TV, a set-top box, a smart vehicle-mounted device, a portablemusic player, a personal digital assistant, a specialized messagingdevice, an intelligent voice interaction device, a portable gamingdevice, and a smart speaker. The terminal and the server may be directlyor indirectly connected in a wired or wireless communication manner.This embodiment of this disclosure does not limit this.

FIG. 3 is a schematic diagram of a structure of a terminal in FIG. 2according to an embodiment of this disclosure. The terminal 400 shown inFIG. 3 includes: at least one first processor 410, a first memory 450,at least one first network interface 420, and a first user interface430. The various components in the terminal 400 are coupled together byusing a first bus system 440. It can be understood that the first bussystem 440 is configured to implement connection and communicationbetween the components. In addition to a data bus, the first bus system440 further includes a power bus, a control bus, and a state signal bus.But, for ease of clear description, all types of buses in FIG. 3 aremarked as the first bus system 440.

The first processor 410 may be processing circuitry, including anintegrated circuit chip having a signal processing capability, such as ageneral purpose processor, a Digital Signal Processor (DSP), or otherprogrammable logic devices, a discrete gate or transistor logic device,and a discrete hardware assembly. The general purpose processor may be amicroprocessor or any conventional processor, and the like.

The first user interface 430 includes one or more first outputapparatuses 431 capable of displaying a media content, including one ormore speakers and/or one or more visual display screens. The first userinterface 430 further includes one or more input apparatuses 432,including a user interface component helpful to typing, such as akeyboard, a mouse, a microphone, a touch display screen, a camera, otherinput buttons, and a control.

The first memory 450 may be removable, unremovable, or anothercombination. An exemplary hardware device includes a solid-state memory,a hard disk drive, an optical disk drive, and the like. The first memory450 alternatively includes one or more storage devices away from thefirst processor 410 in physical positions.

The first memory 450 may be a volatile memory or a non-volatile memory,or may include both a volatile memory and a non-volatile memory. Forexample, the non-volatile memory may be a read only memory (ROM), andthe volatile memory may be a random access memory (RAM). The firstmemory 450 described in this embodiment of this disclosure aims toinclude any other suitable type of memories.

In some embodiments, the first memory 450 can store data to supportvarious operations. Examples of the data include a program, a module,and a data structure or other subsets or supersets, which areexemplified below.

A first operating system 451 includes a system program used forprocessing various basic system services and performing hardware-relatedtasks, for example, a framework layer, a core library layer, a drivelayer, and the like, to implement various basic services and processhardware-based tasks.

A first network communication module 452 is configured to reach otherelectronic devices via one or more (wired or wireless) first networkinterfaces 420. Exemplarily, the first network interfaces 420 include:Bluetooth, Wi-Fi, Universal Serial Bus (USB), and the like.

A first display module 453 is configured to display information via theone or more first output apparatuses 431 (for example, a display screenand speaker) associated with the first user interface 430 (for example,a user interface used for operating peripheral devices and displayingcontents and information).

A first input processing module 454 is configured to detect one or moreuser inputs or interactions from the one or more first input apparatuses432 and translate the detected inputs or interactions.

In some embodiments, the first multimedia stream processing apparatusprovided by this embodiment of this disclosure may be implemented insoftware. FIG. 3 shows the first multimedia stream processing apparatus455 stored in the first memory 450, which can be software in the form ofa program and a plug-in, including following software modules: aninformation obtaining module 4551, an information playing module 4552,and a processing triggering module 4553. These modules are logic, sothat they can be combined in various manners or further dividedaccording to functions achieved. Exemplary functions of the variousmodules will be explained below.

Referring to FIG. 4 , FIG. 4 is a schematic diagram of a structure of aserver in FIG. 2 according to an embodiment of this disclosure. Theserver 200 shown in FIG. 4 includes: at least one second processor 210,a second memory 250, at least one second network interface 220. Thevarious components in the server 200 are coupled together by using asecond bus system 240. It may be understood that the second bus system240 is configured to implement connection and communication betweenthese components. In addition to a data bus, the second bus system 240further includes a power bus, a control bus, and a state signal bus.But, for ease of clear description, all types of buses in FIG. 4 aremarked as the second bus system 240.

The second processor 210 may be processing circuitry, including anintegrated circuit chip having a signal processing capability, such as ageneral purpose processor, a DSP, or other programmable logic devices, adiscrete gate or transistor logic device, and a discrete hardwareassembly. The general purpose processor may be a microprocessor or anyconventional processor, and the like.

The second memory 250 may be removable, unremovable, or anothercombination. An exemplary hardware device includes a solid-state memory,a hard disk drive, an optical disk drive, and the like. The secondmemory 250 alternatively includes one or more storage devices away fromthe second processor 210 in physical positions.

The second memory 250 may be a volatile memory or a non-volatile memory,or may include both a volatile memory and a non-volatile memory. Forexample, the non-volatile memory may be a ROM, and the volatile memorymay be a RAM. The second memory 250 described in this embodiment of thisdisclosure aims to include any other suitable type of memories.

In some embodiments, the second memory 250 can store data to supportvarious operations. Examples of the data include a program, a module,and a data structure or other subsets or supersets, which areexemplified below.

A second operating system 251 includes a system program used forprocessing various basic system services and performing hardware-relatedtasks, for example, a framework layer, a core library layer, a drivelayer, and the like, to implement various basic services and processhardware-based tasks.

A second network communication module 252 is configured to reach otherelectronic devices via one or more (wired or wireless) second networkinterfaces 220. Exemplarily, the second network interfaces 220 include:Bluetooth, Wi-Fi, USB, and the like.

A second input processing module 254 is configured to detect one or moreuser inputs or interactions from the one or more second inputapparatuses and translate the detected inputs or interactions.

In some embodiments, the second multimedia stream processing apparatusprovided by this embodiment of this disclosure may be implemented insoftware. FIG. 4 shows the second multimedia stream processing apparatus255 stored in the second memory 250, which can be software in the formof a program and a plug-in, including following software modules: anevent marking module 2551, an information coding module 2552, a timeobtaining module 2553, and an information transmission module 2554.These modules are logic, so that they can be combined in various mannersor further divided according to functions achieved. The functions of thevarious modules will be explained below.

In some embodiments, the first and second multimedia stream processingapparatuses according to the embodiments of this disclosure may also beimplemented in hardware. For example, the first and second multimediastream processing apparatuses according to the embodiments of thisdisclosure may be processors in the form of a hardware decodingprocessor, and are programmed to implement a multimedia streamprocessing method provided by the embodiments of this disclosure. Forexample, processing circuitry, such as the processor in the form of ahardware decoding processor, may use one or more application-specificintegrated circuits (ASICs), a DSP, a programmable logic device (PLD), acomplex programmable logic device (CPLD), a field-programmable gatearray (FPGA), or other electronic elements.

In some embodiments, the terminal or server may implement the multimediastream processing method provided in this embodiment of this disclosureby running a computer program. For example, the computer program may bea native program or software module in an operating system. The computerprogram may be a native application (APP), which cannot be run until itis installed in the operating system, for example, a livestreaming APPor an instant messaging APP. The computer program may also be a miniprogram, which can be run after being simply downloaded to a browserenvironment. The computer program may also be a mini program that can beembedded into any APP. In summary, the above computer program can be anyform of application program, module, or plug-in.

The multimedia stream processing methods provided by the embodiments ofthis disclosure will be described below in combination with exemplaryapplications and implementations of the player device and the serverdevice provided by the embodiments of this disclosure. In addition, themultimedia stream processing methods provided in the embodiments of thisdisclosure may be applied to various scenarios such as a cloudtechnology, artificial intelligence, intelligent transportation, andvehicles.

FIG. 5 a is a first flowchart of a multimedia stream processing methodaccording to an embodiment of this disclosure. The method will bedescribed in combination with steps shown in FIG. 5 a.

In step S501, a server device performs event marking on a to-be-codedmultimedia stream (or multimedia stream) to obtain at least one eventmarking moment. In an example, event marking is performed on a contentstream to obtain first trigger timing information of a target event inthe content stream.

In this embodiment of this disclosure, the server device includes afunction used for marking an event. Therefore, in order to triggerspecified processing in a player device on the basis of an event in amultimedia stream, the server device performs the event marking on theto-be-coded multimedia stream before coding, so as to mark at least oneevent marking moment corresponding to at least one event in theto-be-coded multimedia stream.

The to-be-coded multimedia stream is a piece of multimedia information,including at least one of an audio stream and a video stream. It may bea live stream or an on-demand stream. This embodiment of this disclosuredoes not limit this. Each event in the at least one event is an event inthe to-be-coded multimedia stream. The at least one event may be of thesame type (for example, events corresponding to big moments such as acontinuous virtual kill event in a game scene and a goal event, wherethe continuous virtual kill event is a “virtual penta kill” event in thegame scene), or may be of different types (for example, a virtual killevent and an event of controlling a virtual object of the other party).This embodiment of this disclosure does not limit this. The at least oneevent corresponds to the at least one event marking moment on aone-to-one basis, so that each event marking moment is a moment at whichan event in the to-be-coded multimedia stream occurs, and a time axis ofthe at least one event marking moment is the same as a time axis of theto-be-coded multimedia stream. When the server device includes an eventmarking device, the server device may achieve event marking processingthrough the event marking device.

In step S502, the server device codes the to-be-coded multimedia streamto obtain the to-be-decoded multimedia stream (or a coded multimediastream). In an example, the content stream is coded to obtain a codedcontent stream.

In this embodiment of this disclosure, the server device furtherincludes a function used for coding. Therefore, the server device codesthe to-be-coded multimedia stream before transmitting the same to theplayer device, so as to improve the transmission efficiency. An obtainedcoding result is the to-be-decoded multimedia stream. That is, theto-be-decoded multimedia stream is a to-be-coded multimedia stream aftercoding. The to-be-decoded multimedia stream corresponds to to-be-codedmultimedia information, and is also a piece of multimedia informationincluding at least one of an audio stream and a video stream. When theserver device also includes a coding device, the server device mayachieve coding processing through the coding device.

The coding processing and the event marking processing are performed inno particular order. The coding processing may be performed first beforethe event marking processing, or the event marking processing may beperformed first before the coding processing, or the coding processingand the event marking processing may be performed in parallel. Thisembodiment of this disclosure is not limited to this. FIG. 5 a showsonly an exemplary execution order. For example, in a livestreamingscene, the event marking processing and the coding processing may beperformed in sequence.

In this embodiment of this disclosure, when the server device may alsoinclude a multimedia stream distribution device, the server device maytransmit the to-be-coded multimedia stream to the coding device and theevent marking device respectively through the multimedia streamdistribution device, so as to enable the coding device to achieve thecoding processing and enable the event marking device to achieve theevent marking processing. S502 may be triggered through the followingprocessing: The server device receives a playing request transmitted bythe player device, and executes S502 in response to the playing request.

In step S503, the server device obtains stream time change informationbetween the to-be-coded multimedia stream and the to-be-decodedmultimedia stream. In an example, stream time change information thatindicates a timing change between the coded content stream and thecontent stream is obtained.

In this embodiment of this disclosure, the server device also records atime change of the to-be-decoded multimedia stream relative to theto-be-coded multimedia stream in the process of obtaining theto-be-decoded multimedia stream, and determines the recorded time changeof the to-be-decoded multimedia stream relative to the to-be-codedmultimedia stream as the stream time change information between theto-be-coded multimedia stream and the to-be-decoded multimedia stream.

The stream time change information may be a time variation between theto-be-coded multimedia stream and the to-be-decoded multimedia stream,or may be a moment of target frame information in the to-be-codedmultimedia stream, a moment of the target frame information in theto-be-decoded multimedia stream, and the like. This embodiment of thisdisclosure does not limit this. The target frame information may beframes having same contents in the to-be-coded multimedia stream and theto-be-decoded multimedia stream, such as a first frame, an end frame, afirst frame or an end frame of a specified clip, and the like. Theserver device may obtain the stream time change information through thecoding device.

In step S504, the server device transmits the to-be-decoded multimediastream to a player device on the basis of the at least one event markingmoment and the stream time change information. In an example, the codedcontent stream is transmitted based on the first trigger timinginformation and the stream time change information, second triggertiming information of the target event in the coded content stream beingdetermined based on the first trigger timing information and the streamtime change information.

In this embodiment of this disclosure, the server device may transmitthe at least one event marking moment, the stream time changeinformation, and the to-be-decoded multimedia stream to the playerdevice to cause the player device to determine at least one eventtriggering moment based on at least one event triggering moment on thebasis of the at least one event marking moment and the stream timechange information. The server device may also determine the at leastone event triggering moment on the basis of the at least one eventmarking moment and the stream time change information, and transmit theat least one event triggering moment and the to-be-decoded multimediastream to the player device. This embodiment of this disclosure does notlimit this.

The server device obtains the at least one event marking moment and thestream time change information to obtain the at least one eventtriggering moment corresponding to at least one event in theto-be-decoded multimedia stream. The processing of obtaining the atleast one event triggering moment on the basis of the at least one eventmarking moment and the stream time change information can be executed byeither the server device or the player device. This embodiment of thisdisclosure does not limit this. After the server device transmits theto-be-decoded multimedia stream to the player device, the player devicealso receives the to-be-decoded multimedia stream.

In step S505, the player device obtains the at least one eventtriggering moment corresponding to the to-be-decoded multimedia stream.In an example, second trigger timing information of the coded contentstream is obtained, the second trigger timing information beingdetermined based on first trigger timing information of the contentstream and stream time change information, the stream time changeinformation indicating a timing change between the coded content streamand the content stream, and the first trigger timing informationindicating a target event in the content stream.

The at least one event triggering moment is obtained by adjusting the atleast one event marking moment on the basis of the stream time changeinformation. The stream time change information refers to a time changebefore and after the coding of the multimedia stream coding, and eachevent marking moment refers to an event moment in a multimedia streambefore coding. The multimedia stream before coding is the to-be-codedmultimedia stream. If the server device transmits the at least one eventmarking moment and the stream time change information to the playerdevice, the player device will receive the at least one event markingmoment and the stream time change information. Therefore, the playerdevice may determine the at least one event triggering moment on thebasis of the at least one event marking moment and the stream timechange information. If the server device transmits the at least oneevent triggering moment to the player device, the player device willreceive the at least one event triggering moment. That is, the playerdevice may adjust the at least one event marking moment by using thestream time change information to obtain the at least one eventtriggering moment, or may obtain the at least one event triggeringmoment by receiving the at least one event triggering moment transmittedby the server device. This embodiment of this disclosure does not limitthis.

In step S506, the player device decodes the to-be-decoded multimediastream to obtain a to-be-played multimedia stream, and plays theto-be-played multimedia stream. In an example, the coded multimediastream is decoded.

In this embodiment of this disclosure, the player device decodes thereceived to-be-decoded multimedia stream to obtain the to-be-playedmultimedia stream, and plays the to-be-played multimedia stream.

The to-be-played multimedia stream is a decoding result of theto-be-decoded multimedia stream, which is a to-be-decoded multimediastream after decoding. Due to different coding manners, the to-be-playedmultimedia stream may be the same as or different from the to-be-codedmultimedia stream. This embodiment of this disclosure does not limitthis.

In step S507, the player device performs, in a case that theto-be-played multimedia stream is played to a target event triggeringmoment, specified processing corresponding to the target eventtriggering moment. In an example, when the decoded content stream isplayed to the target event indicated by the second trigger timinginformation, specified processing corresponding to the target event isperformed.

In this embodiment of this disclosure, in the process that the playerdevice plays the to-be-played multimedia stream, if the to-be-playedmultimedia stream is played to any event triggering moment among the atleast one event triggering moment, it indicates that what is currentlyplayed in the player device is a content of one event (for example, atleast one of a picture and an audio). At this time, the specifiedprocessing at the event triggering moment is triggered, so that theplaying of the content of the event is synchronized with the triggeringof the specified processing. The target event triggering moment is anyone of the at least one event triggering moment.

The server device transmits the to-be-decoded multimedia stream to theplayer device on the basis of the at least one event marking moment andthe stream time change information, so that the player device plays theto-be-played multimedia stream by decoding the to-be-decoded multimediastream, and executes, in a case that the to-be-played multimedia streamis played to the target event triggering moment, the specifiedprocessing corresponding to the target event triggering moment.

For example, referring to FIG. 5 b , FIG. 5 b is a flowchart of anexemplary multimedia stream processing method according to an embodimentof this disclosure. The exemplary multimedia stream processing methoddescribes a process that a game server (referred to as a server device)transmits a live game video stream (referred to as a to-be-codedmultimedia stream) to a game client on a terminal (referred to as aclient device), and a process that the client device plays the receivedlive game video stream and presents a virtual red packet (referred to asspecified processing) when playing a picture of an “virtual penta kill”event (referred to as a target event). The method will be describedbelow in combination with steps shown in FIG. 5 b.

In step S5 b 01, a game server marks a “virtual penta kill” event in alive game video stream to obtain at least one “virtual penta kill” eventmarking moment.

The live game video stream is the to-be-coded multimedia stream in thisembodiment of this disclosure. The process of marking the “virtual pentakill” event is an event marking process. The obtained at least one“virtual penta kill” event marking moment corresponds to the at leastone event marking moment in this embodiment of this disclosure.

In step S5 b 02, the game server codes the live game video stream toobtain a to-be-transmitted live game video stream.

The to-be-transmitted live game video stream is the to-be-decodedmultimedia stream.

In step S5 b 03, the game server obtains stream time change informationbetween the live game video stream and the to-be-transmitted live gamevideo stream.

The process of obtaining the stream time change information between thelive game video stream and the to-be-transmitted live game video streamis the process of obtaining the stream time change information betweenthe to-be-coded multimedia stream and the to-be-decoded multimediastream in this embodiment of this disclosure.

In step S5 b 04, the game server transmits the at least one “virtualpenta kill” event marking moment, the stream time change information,and the to-be-transmitted live game video stream to a game client.

Explanation is made by taking processing of adjusting execution time ofthe game client.

In step S5 b 05, the game client adjusts the at least one “virtual pentakill” event marking moment on the basis of the stream time changeinformation to obtain at least one virtual red packet event triggeringmoment.

If the game server transmits the at least one “virtual penta kill” eventmarking moment, the stream time change information, and theto-be-transmitted live game video stream to the game client, the gameclient will receive the at least one “virtual penta kill” event markingmoment, the stream time change information, and the to-be-transmittedlive game video stream transmitted by the game server. At this time, thegame client adjusts each “virtual penta kill” event marking moment onthe basis of the stream time change information to obtain eachcorresponding virtual red packet event triggering moment. The at leastone virtual red packet event triggering moment is the at least one eventtriggering moment in this embodiment of this disclosure.

In step S5 b 06, the game client plays a live game video by decoding theto-be-transmitted live game video stream.

The process that the game client plays the live game video is theprocess that the server device plays the to-be-played multimedia stream.

In step S5 b 07, when the live game video is played to the at least onevirtual red packet event triggering moment, the game client displays apicture of a corresponding “virtual penta kill” event and displays avirtual packet.

When the live game video is played to the at least one virtual redpacket event triggering moment, that is, when the to-be-playedmultimedia stream is played to the target event triggering moment, thespecified processing performed by the game client is displaying thevirtual red packet. Since the game client device adjusts each “virtualpenta kill” event marking moment on the basis of the stream time changeinformation, a playing moment of the picture of the “virtual penta kill”event is consistent with the adjusted “virtual penta kill” event markingmoment (namely, the virtual red packet event triggering moment).Therefore, it is possible to achieve synchronous displaying of thepicture of the “virtual penta kill” event and the virtual red packet,improving the game video playing effect.

It can be understood that among the at least one event triggering momentobtained by the player device, each event triggering moment is obtainedby adjusting the at least one event marking moment on the basis of thestream time change information, and the stream time change informationrefers to the time change before and after the coding of the multimediastream. Therefore, each event triggering moment obtained by the playerdevice is a moment at which an event in a to-be-decoded live streamoccurs. In this way, in the process of playing the to-be-playedmultimedia stream obtained by decoding the to-be-decoded multimediastream, a played content matches the event triggering moment.Accordingly, if any event triggering moment to which the to-be-playedmultimedia stream is played is just a moment of playing the eventcontent, at this time, the specified processing is triggered, so thatthe accuracy of triggering specified processing can be improved.

FIG. 6 a is a second flowchart of a multimedia stream processing methodprovided according to an embodiment of this disclosure. As shown in FIG.6 a , in this embodiment of this disclosure, S504 may be implemented byS5041. That is, the server device transmits the to-be-decoded multimediastream to the player device on the basis of the at least one eventmarking moment and the stream time change information, including S5041.This step will be explained below.

In step S5041, the server device transmits the at least one eventmarking moment, the stream time change information, and theto-be-decoded multimedia stream to the player device.

In this embodiment of this disclosure, if the server device transmitsthe at least one event marking moment and the stream time changeinformation to the player device, the player device will receive thestream time change information and the at least one event marking momenttransmitted by the server device.

The server device may include a broadcasting service device. At thistime, the server device may broadcast the at least one event markingmoment to each player device through the broadcasting service device.The server device also includes a playing service device. Each playerdevice requests the corresponding coding device to code the to-be-codedmultimedia stream through the playing service device, and obtains,through the playing service device, the to-be-decoded multimedia streamcoded by the corresponding coding device and the stream time changeinformation of the to-be-decoded multimedia stream relative to theto-be-coded multimedia stream.

Correspondingly, in this embodiment of this disclosure, S505 may beimplemented through S5051. That is, the player device obtains the atleast one event triggering moment corresponding to the to-be-decodedmultimedia stream, including S5051. This step will be explained below.

In step S5051, the player device adjusts the at least one event markingmoment on the basis of the stream time change information to obtain theat least one event triggering moment.

In this embodiment of this disclosure, the player device adjusts atleast one actually marked event marking moment using the stream timechange information to be consistent with a change in the time axis ofthe to-be-coded multimedia stream in the coding process, so that thetime axis of at least one adjusted event triggering moment is consistentwith the time axis of the to-be-decoded multimedia stream.

FIG. 6 b is a flowchart of adjusting at least one event marking momentaccording to an embodiment of this disclosure. As shown in FIG. 6 b , inthis embodiment of this disclosure, the stream time change informationincludes a to-be-coded time stamp and a coding time stamp of targetframe information. At this time, S5051 may be implemented through S50511and S50512. That is, the player device adjusts the at least one eventmarking moment on the basis of the stream time change information toobtain the at least one event triggering moment, including S50511 andS50512. The various steps will be explained separately below.

In step S50511, the player device obtains a time variation between thecoding time stamp and the to-be-coded time stamp.

Both the to-be-coded multimedia stream and the to-be-decoded multimediastream include the target frame information. The to-be-coded time stampis a time stamp corresponding to the target frame information in theto-be-coded multimedia stream, and the to-be-coded time stamp is a timestamp corresponding to the target frame information in the to-be-decodedmultimedia stream. The target frame information is frame informationcontaining a same multimedia content in the to-be-coded multimediastream and the to-be-decoded multimedia stream, and the time variationrefers to a relative time difference between the coding time stamp andthe to-be-coded time stamp.

In step S50512, the player device fuses the time variation with eachevent marking moment to obtain the at least one event triggering momentcorresponding to the at least one event marking moment.

In this embodiment of this disclosure, the player device applies theobtained time variation to each event marking moment, to adaptivelyadjust each event marking moment on the basis of a time stamp change ofthe to-be-coded multimedia stream in the coding process, so that eachevent triggering moment obtained through adjustment corresponds to anevent content in the to-be-decoded multimedia stream. Fusing, by theplayer device, the time variation with each event marking moment may beachieved by performing addition or offsetting each event marking momenton the basis of the time variation. This embodiment of this disclosuredoes not limit this.

It can be understood that the server device records the time stamps ofthe target frame information in the multimedia streams before and aftercoding (the to-be-coded multimedia stream is a multimedia stream beforecoding and the to-be-decoded multimedia stream is a multimedia streamafter coding), and transmits the two recorded time stamps to the playerdevice to cause the player device to adaptively adjust each receivedevent marking moment to obtain the at least one event triggering momentwith the time axis consistent with the time axis of the to-be-decodedmultimedia stream. In this way, the consistency between an event contentand an event playing moment in an actual playing process can beimproved, thereby improving the accuracy of triggering a specifiedevent.

FIG. 7 a is a third flowchart of a multimedia stream processing methodprovided according to an embodiment of this disclosure. As shown in FIG.7 a , in this embodiment of this disclosure, S504 and S505 may beimplemented by S508 and S509. That is, the server device transmits theto-be-decoded multimedia stream to the player device on the basis of theat least one event marking moment and the stream time changeinformation, including S508 and S509. The various steps will beexplained below.

In step S508, the server device adjusts the at least one event markingmoment on the basis of the stream time change information to obtain theat least one event triggering moment.

The server device adjusts the at least one event marking moment on thebasis of the stream time change information to obtain the at least oneevent triggering moment. For example, if the stream time changeinformation includes the to-be-coded time stamp and the coding timestamp of the target frame information, the server device obtains thetime variation between the coding time stamp and the to-be-coded timestamp, and fuses the time variation with each event marking moment toobtain the at least one event triggering moment corresponding to the atleast one event marking moment. The corresponding adjustment process issimilar to the processing process described in S50511 and S50512, but anexecutive body is different.

In step S509, the server device transmits the at least one eventtriggering moment and the to-be-decoded multimedia stream to the playerdevice.

In this embodiment of this disclosure, after the server device adjuststhe at least one event marking moment on the basis of the stream timechange information to obtain the at least one event triggering moment,the server device transmits the at least one event triggering moment andthe to-be-decoded multimedia stream to the player device. If the serverdevice transmits the at least one event triggering moment to the playerdevice, the player device will receive the at least one event triggeringmoment transmitted by the server device and corresponding to theto-be-decoded multimedia stream. Furthermore, the at least one eventtriggering moment is obtained by adjusting, by the server device, the atleast one event marking moment on the basis of the stream time changeinformation.

If the server device includes an event marking device, a broadcastingdevice, a coding device, and a playing service device, the playingservice device obtains the at least one event marking moment from theevent marking device through the broadcasting device, obtains the streamtime change information and the to-be-decoded multimedia streams fromthe coding device, adjusts the at least one event marking moment intothe at least one event triggering moment on the basis of the stream timechange information, and transmits the to-be-decoded multimedia streamand the at least one event triggering moment to the player device.

It can be understood that by adjusting the at least one event markingmoment in the server device, the resource consumption of the playerdevice for adjusting the at least one event marking moment can bereduced. Compared to the performance of the player device, theperformance of the server device is better. By implementing theadjustment processing through the server device, the adjustmentefficiency can be improved, thereby improving the multimedia streamprocessing efficiency. In addition, compared with separately adjustingthe at least one event marking moment by a plurality of player devices,adjusting the at least one event marking moment by the server device canreduce the adjustment load and achieve multiplexing of an adjustmentresult, thereby reducing the resource consumption for processing themultimedia stream.

In this embodiment of this disclosure, the specified processing in S507includes at least one of displaying virtual resource information,displaying a specified special effect, inter-cutting multimediainformation, displaying question-and-answer information, and displayingrecommendation information.

The virtual resource information includes at least one of a virtualresource packet and a virtual gift. The player device performs, in acase that the specified processing includes displaying virtual resourceinformation, the specified processing corresponding to the target eventtriggering moment, which includes: The player device displays thevirtual resource information in a case that the target event triggeringmoment refers to a moment at which a virtual scene event occurs.Moreover, after the player device displays the virtual resource packet,the multimedia stream processing method further includes: the playerdevice displays an obtained target virtual resource, such as an openedvirtual red packet and a grabbed virtual gift, in response to atriggering operation performed on the virtual resource packet.

A specified special effect played by the player device may be a specialeffect associated with at least one event. For example, if the event isa goal event, a special effect of applause is displayed. If the event isa continuous virtual kill event, a special effect of spreading virtualcoins. If the event is a game winning streak, an animated special effectof congratulating the winning streak is displayed. Or the like. Inaddition, if the to-be-played multimedia stream includes subtitleinformation, the subtitle information associated with the event contentmay be marked and displayed when a target event is triggered.

The multimedia information inter-cut by the player device is differentfrom the to-be-played multimedia stream, such as advertisements,messages, and announcement information.

After the player device displays interaction information such asquestion-and-answer information and recommendation information, themultimedia stream processing method further includes: the player deviceperforms, in response to an interaction operation performed on theinteraction information, processing corresponding to the interactionoperation, for example, application switching and played informationswitching.

In this embodiment of this disclosure, the event marking in S501 isachieved through at least one of a marking operation, a neural networkmodel, and marker information. The marking operation refers to anoperation used for determining an event marking moment. The neuralnetwork model is used for determining the event marking moment on thebasis of a content of the to-be-played multimedia stream, and the neuralnetwork model is trained on the basis of a multimedia stream sample anda marked event moment. The marker information is used for representingthe event marking moment, and may be information carried in theto-be-coded multimedia stream.

FIG. 7 b is a schematic diagram of obtaining stream time changeinformation according to an embodiment of this disclosure. As shown inFIG. 7 b , in this embodiment of this disclosure, S503 may beimplemented through S5031 to S5034. That is, the server device obtainsthe stream time change information between the to-be-coded multimediastream and the to-be-decoded multimedia stream, including S5031 toS5034. The various steps will be explained below.

In step S5031, the server device obtains a to-be-coded time stampcorresponding to target frame information of the to-be-coded multimediastream.

In this embodiment of this disclosure, the coding device in the serverdevice records a time stamp corresponding to the target frameinformation in the to-be-coded multimedia stream after obtaining theto-be-coded multimedia stream distributed by the multimedia streamdistribution device, thereby obtaining the to-be-coded time stamp.

In step S5032, the server device obtains a coding manner of theto-be-decoded multimedia stream.

The coding manner includes any one of a first specified coding manner, asecond specified coding manner, and a third specified coding manner. Thefirst specified coding manner refers to a manner of not changing apresentation time stamp in the coding process. The second specifiedcoding manner refers to a manner of determining a presentation timestamp on the basis of synchronizing time information (such as timeinformation provided by a synchronous clock) in the coding process. Thethird specified coding manner refers to a manner of determining apresentation time stamp on the basis of specified time information (suchas 0 milliseconds) in the coding process.

In step S5033, the server device determines a coding time stamp of thetarget frame information in the to-be-decoded multimedia stream on thebasis of the coding manner.

In this embodiment of this disclosure, the to-be-coded time stamp isdetermined as the coding time stamp of the target frame information inthe to-be-decoded multimedia stream in a case that the coding manner isthe first specified coding manner A synchronizing time stamp is obtainedin a case that the coding manner is the second specified coding manner,and the synchronizing time stamp is determined as the coding time stampof the target frame information in the to-be-decoded multimedia stream,the synchronizing time stamp being the synchronizing time information. Aspecified time stamp is determined as the coding time stamp of thetarget frame information in the to-be-decoded multimedia stream in acase that the coding manner is the third specified coding manner, thespecified time stamp being specified time information.

In step S5034, the server device determines the to-be-coded time stampand the coding time stamp as the stream time change information betweenthe to-be-coded multimedia stream and the to-be-decoded multimediastream.

The server device transmits the to-be-coded time stamp and the codingtime stamp as the stream time change information to the player device tocause the player device to adjust each event marking moment on the basisof the to-be-coded time stamp and the coding time stamp in the streamtime change information. Or, the server device may take the to-be-codedtime stamp and the coding time stamp as the stream time changeinformation, and adjust each event marking moment on the basis of theto-be-coded time stamp and the coding time stamp in the stream timechange information to transmit each event triggering moment to theplayer device.

In this embodiment of this disclosure, the server device furtherincludes a time stamp synchronization device, configured to forward thestream time change information obtained by the coding device to theplaying service device, and also configured to provide the synchronizingtime stamp to the coding device to cause the coding device to code theto-be-coded multimedia stream on the basis of the synchronizing timestamp.

An exemplary application of this embodiment of this disclosure in onepractical application scenario will be described below. This exemplaryapplication describes that in a game livestreaming process, a terminal(referred to as the player device) accurately triggers displaying of avirtual resource packet (referred to as the specified processing) duringplaying of audios and videos (referred to as the to-be-played multimediastream) while playing pictures of excellent events (referred to as thetarget event triggering moment).

FIG. 8 is a schematic diagram of an exemplary event picture according toan embodiment of this disclosure. As shown in FIG. 8 , in a virtualscene 8-1, a virtual object 8-11 has virtually killed five virtualobjects of an opposing team within specified time, which triggers the“virtual penta kill” event.

Based on FIG. 8 , FIG. 9 is a schematic diagram illustrating that anexemplary time stamp changes according to an embodiment of thisdisclosure. As shown in FIG. 9 , a process of changing a time stampcorresponding to an event in a multimedia stream by coding is described:In a live stream 9-1 before coding, a time stamp 9-11 (referred to asthe event marking moment) of the “virtual penta kill” event in FIG. 8 isexpressed as PTS_0 in FIG. 9 . In a live stream 9-2 (referred to as theevent triggering moment) obtained by coding the live stream 9-1, a timestamp 9-21 of the “virtual penta kill” event in FIG. 8 is expressed asPTS_1 in FIG. 9 . PTS_0 and PTS_1 correspond to different time. That is,coding will change a time stamp of frame information in a live stream.An event moment (for example, the time stamp 9-11) before coding can beadjusted on the basis of a change between the time stamps before andafter coding, so that an adjusted event moment (for example, the timestamp 9-21) more precisely matches an event occurrence moment in thecoded live stream. The adjustment process will be described in detailbelow.

Referring to FIG. 10 , FIG. 10 is a schematic diagram of an architectureof a live stream system according to an embodiment of this disclosure.As shown in FIG. 10 , when a live stream 10-1 (expressed as a livestream ID_0, referred to as the to-be-coded multimedia stream) reaches astream distribution server 10-2 (referred to as the multimedia streamdistribution device), the stream distribution server 10-2 performsmulti-path distribution on the live stream 10-1. In one path, the livestream 10-1 is transmitted to an event management server 10-3 (referredto as the event marking device). In another path, the live stream 10-1is transmitted to n (n is a positive integer greater than 1) codingservers 10-4 (exemplarily showing a coding server 10-41 to a codingserver 10-4 n, referred to as the coding device).

The event management server 10-3 performs event marking on the livestream 10-1 to obtain each marked moment on a time axis of the livestream 10-1 (for example, the “virtual penta kill” moment or otherexciting moments, referred to as the event marking moment). The eventmarking may be achieved by receiving an operation of a user (referred toas the marking operation), or may be achieved by artificial intelligence(for example, the neural network model), or may be directly obtained onthe basis of indication information transmitted by a game server (forexample, information transmitted synchronously with the live stream10-1, referred to as the marker information). The event managementserver 10-3 transmits each obtained marking moment to a broadcastingserver 10-5 (referred to as the broadcasting device).

The coding servers 10-4 are configured to transmit, when there aredifferences in the network condition and the playing capability, livestreams that best matches a network condition and a playing capabilityto a terminal of the user, to improve the playing fluency and otherplaying effects. Each coding server i among the n coding servers 10-4 (iis a positive integer variable greater than or equal to 1 and less thanor equal to n, namely, any one of the coding server 10-41 to the codingserver 10-4 n) is configured to code the live stream 10-1 into differentlive streams (live stream ID_1 to live stream ID_n). Each coding serveri among the coding servers 10-4 implements coding on the basis of atleast one of different resolutions, coding formats (such as codingformat H.265 and coding format H.264), and code rates. In addition, eachcoding server i may also correspond to one game livestreamingapplication.

For example, FIG. 11 is a schematic diagram of a coding result of anexemplary live stream according to an embodiment of this disclosure. Asshown in FIG. 11 , eight different live streams 11-1 output by the livestream 10-1 in FIG. 10 subjected to the coding processing 11-2 include a1080P-definition live stream at a coding format H.265, a 720P-definitionlive stream at a coding format H.265, a 540P-definition live stream at acoding format H.265, a 360P-definition live stream at a coding formatH.265, a 1080P-definition live stream at a coding format H.264, a720P-definition live stream at a coding format H.264, a 540P-definitionlive stream at a coding format H.264, and 360P-definition live stream ata coding format H.264.

Based on FIG. 11 , FIG. 12 is a schematic diagram of an exemplaryapplication scenario of obtaining different live streams according to anembodiment of this disclosure. As shown in FIG. 12 , it can bedetermined on the basis of a region 12-11 on a page 12-1 that a livestream played on the page 12-1 is a live stream with “high definition of540P”, and a live stream with any definition of “Blue ray 1080P”, “superdefinition 720P”, and “standard definition 360P” can be selected forplaying. A terminal corresponding to the page 12-1 supports a livestream at the coding format H.264, so “standard definition 360P”displayed on the page 12-1 corresponds to the 360P-definition livestream at the coding format H.264 in FIG. 12 ; “high definition 540P”displayed on the page 12-1 corresponds to the 540P-definition livestream at the coding format H.264 in FIG. 12 ; “super definition 720P”displayed on the page 12-1 corresponds to the 720P-definition livestream at the coding format H.264 in FIG. 12 ; and “Blue ray 1080P”displayed on the page 12-1 corresponds to the 1080P-definition livestream at the coding format H.264 in FIG. 12 .

Each coding server i codes the live stream ID_0 to obtain a live streamID_i. If the coding server i may thoroughly transmit the live streamID_0, without changing a Presentation Time Stamp (PTS) corresponding toframe information of the live stream ID_0 (corresponding to the firstspecified coding manner), the live stream ID_i and the live stream ID_0have the same time axis. Thus, a presentation time stamp PTS_is(referred to as a coding time stamp) corresponding to start frameinformation (namely, at least one of a first frame of picture and firstframe of audio that the live stream ID_0 reaches the coding server i,referred to as the target frame information) of the live stream ID_i isthe same as a playing time stamp PTS_i0 (referred to as the to-be-codedtime stamp) corresponding to stat frame information of the live streamID_0. At this time, the marking moment does not change. In addition, thecoding server i may also take synchronizing time (referred to as thesynchronizing time stamp) provided by a time stamp synchronizing server10-6 (referred to as the time stamp synchronization device) as thepresentation time stamp PTS_is corresponding to the start frameinformation of the live stream ID_i (corresponding to the secondspecified coding manner). The coding server i may also set thepresentation time stamp PTS_is (corresponding to a specified time stamp,for example, 0) corresponding to the start frame information of the livestream ID_i by self (corresponding to the third specified codingmanner). Therefore, in this embodiment of this disclosure, whenreceiving the live stream ID_0 distributed by the stream distributionserver 10-2, the coding server i records the presentation time stampPTS_i0 corresponding to the start frame information of the live streamID_0, forms a time stamp pair <PTS_i0, PTS_is> (referred to as thestream time change information) by the presentation time stamp PTS_i0and the presentation time stamp PTS_is corresponding to the start frameinformation of the live stream ID_i, and transmits the time stamp pair<PTS_i0, PTS_is> to the time stamp synchronization server 10-6. If i isgreater than or equal to 0 and less than or equal to n, outputscorresponding to the coding server 10-41 to the coding server 10-4 n are(ID_1, PTS_10, PTS_1s), (ID_2, PTS-20, PTS_2s), . . . , (ID_n, PTS_n0,PTS_ns). Any one of the live stream ID_1 to the live stream ID_n may betransmitted to a terminal 10-7 through distribution by a contentdistribution network (CDN) or through peer-to-peer (P2P) services.

The broadcast server 10-5 broadcasts each marking moment to the terminal10-7. The playing server 10-8 (referred to as the playing servicedevice) obtains the time stamp pair <PTS_i0, PTS_is> from the time stampsynchronization server 10-6, and transmits the same to the terminal10-7. The playing server 10-8 obtains a coded live stream address fromthe coding server 10-4, and transmits the same to the terminal 10-7.Thus, the terminal 10-7 may adjust each marking moment on the basis ofthe time stamp pair, and obtain the live stream ID_i from the codingserver 10-4 on the basis of the live stream address. For example, if themarking moment is PTS_ID0_t, the marking moment is the moment of thelive stream ID_i PTS_IDi_t=PTS_ID0_t+(PTS_is−PTS_i0). When the terminalplays a content to PTS_IDi_t, a corresponding processing logic isactivated, thus triggering the specified processing.

For FIG. 10 , the processing process of the live stream includes aninformation interaction stream (referred to as a time stamp pairinteraction processing stream), a live stream processing stream, and anevent stream (referred to as a processing stream at the marking moment).A correspondence relationship between time stamps PTS_x0 and PTS_y0 oftwo pieces of frame information in the live stream ID_0 and time stampsPTS_xi and PTS_yi of two pieces of frame information in the live streamID_i is as follows: PTS_yi−PTS_xi=PTS_y0−PTS_x0. A live contentcorresponding to the time stamp PTS_x0 is the same as a live contentcorresponding to the time stamp PTS_xi, and a live content correspondingto the time stamp PTS_y0 is the same as a live content corresponding tothe time stamp PTS_yi.

A detailed explanation will be made below to interactions between thestream distribution server 10-2, the event management server 10-3, thecoding server 10-4, the broadcasting server 10-5, the time stampsynchronization server 10-6, the terminal 10-7, and the playing serviceserver 10-8 in FIG. 10 . FIG. 13 is a schematic diagram of an exemplarylive stream interaction according to an embodiment of this disclosure.As shown in FIG. 13 , the schematic diagram of the exemplary live streaminteraction includes S1301 to S1309. The various steps will be explainedbelow.

In step S1301, the stream distribution server 10-2 distributes a livegame stream to the event management server 10-3.

The live game stream is the live stream ID_0 in FIG. 10 .

In step S1302, the event management server 10-3 transmits an excitingmoment to the broadcasting server 10-5.

The event management server 10-3 performs event marking on the livestream ID_0 to obtain each exciting moment (namely, each marking momentin FIG. 10 ), and transmits each exciting moment to the broadcastingserver 10-5.

In step S1303, the terminal 10-7 transmits a live stream request(referred to as a playing request) to the playing server 10-8.

In step S1304, the stream distribution server 10-2 distributes the livegame stream to the coding server 10-4.

The playing server 10-8 enables, in response to the live stream request,the coding server i of the coding server 10-4 to perform codingprocessing, thereby triggering execution of S1305.

In step S1305, the coding server 10-4 transmits a time stamp pair to thetime stamp synchronization server 10-6.

The coding server 10-4 codes the live game stream (the live stream ID_0)to obtain a coded live game stream (the live stream ID_i); and transmitsthe time stamp pair <PTS_i0,PTS_is> formed by the time stampcorresponding to the start frame information of the live stream ID_0 inresponse to the live stream request and the time stamp corresponding tothe start frame information of the live stream ID_i to the time stampsynchronization server 10-6.

In step S1306, the time stamp synchronization server 10-6 transmits thetime stamp pair to the terminal 10-7.

It may be that the time stamp synchronization server 10-6 transmits thetime stamp pair to the terminal 10-7 through the playing server 10-8.

In step S1307, the broadcasting server 10-5 broadcasts the excitingmoment to the terminal 10-7.

In step S1308, the coding server 10-4 transmits the coded live gamestream to the terminal 10-7.

The live stream address may be transmitted to the terminal 10-7 inresponse to the live stream request, so that the coding server 10-4transmits, to the terminal 10-7, the coded live game stream (namely, thelive stream ID_i in FIG. 10 ) matching the live stream address.

In step S1309, the terminal 10-7 plays the coded live game stream, anddisplays a virtual red packet or a virtual gift when a picture (forexample, a video picture of “virtual penta kill” appears) of theexciting moment is played.

It can be understood that the embodiments of this disclosure record thechange in the time stamps during the coding, and adjusts the excitingmoment on the basis of this change. In the coded live stream, anaccurate correspondence relationship between an event and time isestablished, which can help ensure that a response corresponding to theevent is made only when a specific picture appears. Even in a case of anuncontrollable network latency, the consistency of the response logicsof the various terminals can be also ensured, and a matching degreebetween specified processing and a played content can be increased. Inaddition, the processing performed by this embodiment of this disclosureon the live game stream makes the coding of the live game stream beuniversal. Moreover, in the processing of the live game stream in thisembodiment of this disclosure, the processing of the event and theprocessing of the live game stream are decoupled, and will not affecteach other. Thus, the universality is relatively high.

An exemplary structure of a first multimedia stream processing apparatus455 implemented as a software module and provided by an embodiment ofthis disclosure is continuously described below. In some embodiments, asshown in FIG. 3 , software modules stored in the first multimedia streamprocessing apparatus 455 of a first memory 450 may include aninformation obtaining module 4551, an information playing module 4552,and processing triggering module 4553.

The information obtaining module 4551 is configured to receive ato-be-decoded multimedia stream transmitted by a server device, andobtain at least one event triggering moment corresponding to theto-be-decoded multimedia stream, the at least one event triggeringmoment being obtained by adjusting at least one event marking moment onthe basis of stream time change information, the stream time changeinformation referring to a time change before and after coding of amultimedia stream, and each event marking moment referring to an eventmoment in the multimedia stream before coding.

The information playing module 4552 is configured to decode theto-be-decoded multimedia stream to obtain a to-be-played multimediastream, and play the to-be-played multimedia stream.

The processing triggering module 4553 is configured to perform, in acase that the to-be-played multimedia stream is played to a target eventtriggering moment, specified processing corresponding to the targetevent triggering moment, the target event triggering moment being anyone of the at least one event triggering moment.

In this embodiment of this disclosure, the information playing module4552 is also configured to receive the stream time change informationand the at least one event marking moment transmitted by the serverdevice; and adjust the at least one event marking moment on the basis ofthe stream time change information to obtain the at least one eventtriggering moment.

In this embodiment of this disclosure, the stream time changeinformation includes a to-be-coded time stamp and a coding time stamp oftarget frame information. The information playing module 4552 is alsoconfigured to obtain a time variation between the coding time stamp andthe to-be-coded time stamp; and fuse the time variation with each eventmarking moment to obtain the at least one event triggering momentcorresponding to the at least one event marking moment.

In this embodiment of this disclosure, the information playing module4552 is also configured to receive the at least one event triggeringmoment transmitted by the server device, the at least one eventtriggering moment corresponding to the to-be-decoded multimedia stream,and the at least one event triggering moment being adjusted by theserver device.

In this embodiment of this disclosure, the specified processing includesat least one of displaying virtual resource information, displaying aspecified special effect, inter-cutting multimedia information,displaying question-and-answer information, and displayingrecommendation information.

The embodiments of this disclosure further provide a computer programproduct or a computer program. The computer program product or thecomputer program includes computer-executable instructions stored in acomputer-readable storage medium. A first processor of an electronicdevice (referred to as the player device) reads the computer-executableinstructions from the computer-readable storage medium. The firstprocessor executes the computer-executable instructions to cause theelectronic device to implement the multimedia stream processing methodapplied to the player device according to the embodiments of thisdisclosure.

An exemplary structure of a second multimedia stream processingapparatus 255 implemented as a software module and provided by anembodiment of this disclosure is continuously described below. In someembodiments, as shown in FIG. 4 , software modules stored in the secondmultimedia stream processing apparatus 255 of a second memory 450 mayinclude an event marking module 2551, an information coding module 2552,a time obtaining module 2553, and an information transmission module2554.

The event marking module 2551 is configured to perform event marking ona to-be-coded multimedia stream to obtain at least one event markingmoment.

The information coding module 2552 is configured to code the to-be-codedmultimedia stream to obtain a to-be-decoded multimedia stream.

The time obtaining module 2553 is configured to obtain stream timechange information between the to-be-coded multimedia stream and theto-be-decoded multimedia stream.

The information transmission module 2554 is configured to transmit theto-be-decoded multimedia stream to a player device on the basis of theat least one event marking moment and the stream time changeinformation, the player device being configured to play a to-be-playedmultimedia stream by decoding the to-be-decoded multimedia stream, andperform specified processing in a case that the to-be-played multimediastream is played to a target event triggering moment, the specifiedprocessing corresponding to the target event triggering moment, thetarget event triggering moment being any one of the at least one eventtriggering moment, and the at least one event triggering moment beingobtained by adjusting the at least one event marking moment on the basisof the stream time change information.

In this embodiment of this disclosure, the information transmissionmodule 2554 is also configured to transmit the at least one eventmarking moment, the stream time change information, and theto-be-decoded multimedia stream to the player device, the player devicebeing configured to adjust the at least one event marking moment intothe at least one event triggering moment on the basis of the stream timechange information.

In this embodiment of this disclosure, the information transmissionmodule 2554 is also configured to adjust the at least one event markingmoment on the basis of the stream time change information to obtain theat least one event triggering moment; and transmit the at least oneevent triggering moment and the to-be-decoded multimedia stream to theplayer device.

In this embodiment of this disclosure, the event marking is achieved byat least one of a marking operation, a neural network model, and markerinformation; the marking operation refers to an operation fordetermining the event marking moment; the neural network model is usedfor determining the event marking moment on the basis of a content ofthe to-be-played multimedia stream, and the neural network model istrained on the basis of a multimedia stream sample and a marked eventmoment; and the marker information refers to information for marking theevent marking moment.

In this embodiment of this disclosure, the time obtaining module 2553 isalso configured to obtain a to-be-coded time stamp corresponding totarget frame information of the to-be-coded multimedia stream; obtain acoding manner of the to-be-decoded multimedia stream; determine a codingtime stamp of the target frame information in the to-be-decodedmultimedia stream on the basis of the coding manner; and determine theto-be-coded time stamp and the coding time stamp as the stream timechange information between the to-be-coded multimedia stream and theto-be-decoded multimedia stream.

In this embodiment of this disclosure, the time obtaining module 2553 isalso configured to determine the to-be-coded time stamp as the codingtime stamp of the target frame information in the to-be-decodedmultimedia stream in a case that the coding manner is a first specifiedcoding manner, the first specified coding manner referring to a mannerof not changing a presentation time stamp in a coding process; obtain asynchronizing time stamp in a case that the coding manner is a secondspecified coding manner, and determine the synchronizing time stamp asthe coding time stamp of the target frame information in theto-be-decoded multimedia stream, the second specified coding mannerreferring to a manner of determining a presentation time stamp on thebasis of synchronizing time information in a coding process; anddetermine a specified time stamp as the coding time stamp of the targetframe information in the to-be-decoded multimedia stream in a case thatthe coding manner is a third specified coding manner, the thirdspecified coding manner referring to a manner of determining apresentation time stamp on the basis of specified time information in acoding process.

The embodiments of this disclosure further provide a computer programproduct or a computer program. The computer program product or thecomputer program includes computer-executable instructions stored in acomputer-readable storage medium. A second processor of an electronicdevice (referred to as the server device) reads the computer-executableinstructions from the computer-readable storage medium. The secondprocessor executes the computer-executable instructions to cause theelectronic device to implement the multimedia stream processing methodapplied to the server device according to the embodiments of thisdisclosure.

The embodiments of this disclosure provide a computer-readable storagemedium, storing computer-executable instructions. When thecomputer-executable instructions are executed by a first processor, thefirst processor is caused to implement the multimedia stream processingmethod applied to the player device according to the embodiments of thisdisclosure. Or, when the computer-executable instructions are executedby a second processor, the second processor is caused to implement themultimedia stream processing method applied to the server deviceaccording to the embodiments of this disclosure, for example, themultimedia stream processing method as shown in FIG. 5 a.

In some embodiments, the computer-readable storage medium, such as anon-transitory computer-readable storage medium, may be a ferroelectricrandom access memory (FRAM), a read only memory (ROM), a programmableROM (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), aflash memory, a magnetic surface memory, an optical disk, a CD-ROM, orthe like. The computer-readable storage medium may include one or anycombination of the aforementioned memories.

In some embodiments, the computer-executable instructions may be in theform of programs, software, software modules, scripts, or codes, writtenin any form of programming language (including a compiling orinterpreting language, or a declarative or procedural language), and maybe deployed in any form, including being deployed as standalone programsor as modules, components, subroutines, or other units suitable for usein computing environments.

For example, the computer-executable instructions may, but may notnecessarily correspond to files in a file system, and may be stored aspart of a file that stores other programs or data, such as in one ormore scripts stored in a Hyper Text Markup Language (HTML) document, ina single file dedicated to a discussed program, or in multiplecollaborative files (such as files that store one or more modules,subroutines, or codes).

For example, the computer-executable instructions may be deployed asbeing executed on one electronic device (at this time, this electronicdevice is the player device and the server device), or being executed onmultiple electronic devices located at the same site (at this time, themultiple electronic devices located at the same site are the playerdevice and the server device), or being executed on multiple electronicdevices distributed at multiple sites and interconnected by acommunication network (at this time, the multiple electronic devicesdistributed at the multiple sites and interconnected by thecommunication network are the player device and the server device).

The term module (and other similar terms such as unit, submodule, etc.)in this disclosure may refer to a software module, a hardware module, ora combination thereof. A software module (e.g., computer program) may bedeveloped using a computer programming language. A hardware module maybe implemented using processing circuitry and/or memory. Each module canbe implemented using one or more processors (or processors and memory).Likewise, a processor (or processors and memory) can be used toimplement one or more modules. Moreover, each module can be part of anoverall module that includes the functionalities of the module.

It can be understood that in the embodiments of this disclosure, whererelevant data such as multimedia streams is involved, when theembodiments of this disclosure are applied to a specific product ortechnology, it is necessary to obtain permissions or agreements ofusers, and the collection, use and processing of the relevant data arerequired to comply with the relevant laws, regulations and standards ofrelevant countries and districts.

In an example, in this embodiment of this disclosure, among the at leastone event triggering moment obtained by the player device, each eventtriggering moment is obtained by adjusting the at least one eventmarking moment on the basis of the stream time change information, andthe stream time change information refers to the time change before andafter the coding of the live stream. Therefore, each event triggeringmoment obtained by the player device is a moment at which an event in ato-be-decoded live stream occurs. In this way, in the to-be-playedmultimedia stream obtained by decoding the to-be-decoded multimediastream, the event content matches the event triggering moment.Accordingly, at any event triggering moment to which the to-be-playedmultimedia stream is played, the specified processing corresponding tothe event triggering moment can be accurately triggered, so that theaccuracy of triggering specified processing can be improved. Inaddition, the multimedia stream processing methods provided by theembodiments of this disclosure can improve the universality oftriggering specified processing.

The foregoing descriptions are merely exemplary embodiments of thisdisclosure and are not intended to limit the scope of this disclosure.Any modification, equivalent replacement, and improvement made withinthe spirit and scope of this disclosure shall fall within the scope ofthis disclosure.

What is claimed is:
 1. A method for content stream processing, themethod comprising: receiving a coded content stream of a content streamfrom a server; obtaining second trigger timing information of the codedcontent stream, the second trigger timing information being determinedbased on first trigger timing information of the content stream andstream time change information, the stream time change informationindicating a timing change between the coded content stream and thecontent stream, and the first trigger timing information indicating atarget event in the content stream; decoding the coded content stream;playing back the decoded content stream; and performing, when thedecoded content stream is played to the target event indicated by thesecond trigger timing information, specified processing corresponding tothe target event.
 2. The method according to claim 1, wherein theobtaining the second trigger timing information comprises: receiving thestream time change information and the first trigger timing informationfrom the server; and adjusting an event timing indicated by the firsttrigger timing information based on the stream time change informationto obtain an event timing indicated by the second trigger timinginformation.
 3. The method according to claim 2, wherein the stream timechange information includes a first time stamp and a second time stampof target frame information; and the adjusting the event timingindicated by the first trigger timing information comprises: obtaining atime variation between the second time stamp and the first time stamp;and adjusting the event timing indicated by the first trigger timinginformation according to the time variation.
 4. The method according toclaim 1, wherein the obtaining the second trigger timing informationcomprises: receiving the second trigger timing information from theserver, the second trigger timing information being determined by theserver.
 5. The method according to claim 1, wherein the specifiedprocessing includes displaying secondary content.
 6. The methodaccording to claim 1, wherein the specified processing includes at leastone of displaying virtual resource information, displaying a specifiedspecial effect, inter-cutting content information, or displayinginformation.
 7. A method for content stream processing, the methodcomprising: performing event marking on a content stream to obtain firsttrigger timing information of a target event in the content stream;coding the content stream to obtain a coded content stream; obtainingstream time change information that indicates a timing change betweenthe coded content stream and the content stream; and transmitting thecoded content stream based on the first trigger timing information andthe stream time change information, second trigger timing information ofthe target event in the coded content stream being determined based onthe first trigger timing information and the stream time changeinformation.
 8. The method according to claim 7, wherein thetransmitting the coded content stream comprises: transmitting the firsttrigger timing information, the stream time change information, and thecoded content stream.
 9. The method according to claim 7, wherein thetransmitting the coded content stream comprises: adjusting an eventtiming indicated by the first trigger timing information based on thestream time change information to determine an event timing indicated bythe second trigger timing information; and transmitting the secondtrigger timing information and the coded content stream.
 10. The methodaccording to claim 7, wherein the event marking is performed by a neuralnetwork model, the neural network model being configured to determinethe first trigger timing information based on content of the contentstream.
 11. The method according to claim 7, wherein the stream timechange information includes a first time stamp corresponding to targetframe information of the content stream and a second time stampcorresponding to the target frame information of the coded contentstream to indicate the timing change.
 12. The method according to claim11, wherein the obtaining the stream time change information comprises:determining the first time stamp as the second time stamp correspondingto the target frame information of the coded content stream based on atype of the coding of the content stream being determined as a firstspecified coding type.
 13. The method according to claim 11, wherein theobtaining the stream time change information comprises: determining thesecond time stamp corresponding to the target frame information of thecoded content stream according to synchronization time information basedon a type of the coding of the content stream being determined as asecond specified coding type.
 14. The method according to claim 11,wherein the obtaining the stream time change information comprises:determining the second time stamp corresponding to the target frameinformation of the coded content stream according to time unitconversion information based on a type of the coding of the contentstream being determined as a third specified coding type.
 15. Aninformation processing apparatus, comprising: processing circuitryconfigured to: receive a coded content stream of a content stream from aserver; obtain second trigger timing information of the coded contentstream, the second trigger timing information being determined based onfirst trigger timing information of the content stream and stream timechange information, the stream time change information indicating atiming change between the coded content stream and the content stream,and the first trigger timing information indicating a target event inthe content stream; decode the coded content stream; play back thedecoded content stream; and perform, when the decoded content stream isplayed to the target event indicated by the second trigger timinginformation, specified processing corresponding to the target event. 16.The information processing apparatus according to claim 15, wherein theprocessing circuitry is configured to: receive the stream time changeinformation and the first trigger timing information from the server;and adjust an event timing indicated by the first trigger timinginformation based on the stream time change information to obtain anevent timing indicated by the second trigger timing information.
 17. Theinformation processing apparatus according to claim 16, wherein thestream time change information includes a first time stamp and a secondtime stamp of target frame information; and the processing circuitry isconfigured to: obtain a time variation between the second time stamp andthe first time stamp; and adjust the event timing indicated by the firsttrigger timing information according to the time variation.
 18. Theinformation processing apparatus according to claim 15, wherein theprocessing circuitry is configured to: receive the second trigger timinginformation from the server, the second trigger timing information beingdetermined by the server.
 19. A non-transitory computer-readable storagemedium, storing instructions, which when executed by a processor causethe processor to perform the method according to claim
 1. 20. Anon-transitory computer-readable storage medium, storing instructions,which when executed by a processor cause the processor to perform themethod according to claim 6.